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Abstract. Assessing student presentations can be made more reliable with video- 
recording and post-performance rating. Further, self assessment and peer assessment 
can aid in the learning process by students when using specific, easy-to-understand 
rubrics. A ten-year action research study involved video-recorded performance 
assessment tasks using a free, open-source Moodle module developed by Sapporo 
Gakuin University. The Video Assessment Module (VAM) allowed teachers to video 
record English presentations and upload them to the module for students for self and 
peer assessment on specific rubrics using qualitative and quantitative criteria. When 
compared to paper rubrics, the VAM reduced teacher management time and students 
could use out-of-class time to assess asynchronously without time pressure. Results 
showed that there was a higher difference in teacher variance for self assessment 
when compared to teacher variance with peer assessment. Qualitative and 
quantitative results reported value in using the tool by both students and teachers. 
This study also showed that students can be trained to use online rubrics to score 
presentations efficiently, giving further validity for using and developing online 


modules for video assessment. 


Keywords: video assessment, presentation skills, performance assessment, rubrics, 
Moodle, self assessment, peer assessment. 


1. Introduction 


Inperformance-based learning, comprehensively assessing largenumbers of students 
on given individual performances has long been a tedious and even unproductive 
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quest for educators throughout second language education. Furthermore, Gardner 
(2012) proposed that for learners to learn from the assessment, they needed to be 
part of the process. This led to practices such as self and peer assessment, both in 
score-giving and qualitative feedback. 


Nicol, Thomson, and Breslin (2014) surveyed peer feedback research and observed 
that effects on both the receiver of peer feedback and the giver of peer feedback 
need to be examined. Based on these principles, the main research question of this 
study was whether online tools could be used and developed to aid in the process 
of post-performance self and peer assessment. Secondly, could participation in 
this type of self and peer assessment aid in improving future English as a Foreign 
Language (EFL) presentation performance for students? Other pertinent research 
questions such as the use of other feedback tools (paper, face-to-face), and timing 
of assessing (real time versus post-performance) are summarized in earlier action 
research by Rian, Hinkelman, and McGarty (2012), Rian, Hinkelman, and Cotter 
(2015), and Hinkelman and Cotter (2018). However, for the purpose of this study, 
results pertaining to the 2019 development cycle of online video assessment tools 
will be addressed. 


2. Method 


2.1. Course background 


Each year, course participants comprise 50-60 second year English major students 
and two teachers at Sapporo Gakuin University. Students engage in a compulsory 
oral English presentation skills class titled ‘Oral Communication C’. During 
weekly 90 minute classes over a single semester of 15 weeks, students were 
required to prepare for and deliver five presentations on varying themes such as 
giving instructions (speech to inform) or Hokkaido sightseeing recommendations 
(speech to persuade). Participation levels were high in these top two levels of 
classes (2019 n=34) with 87% in-class attendance rate and an average rate of nine 
out of ten out-of-class homework quizzes. The research question involved whether 
students could be trained to assess using online rubrics and whether they found 
value in the process. 


After the decision in 2009 to move from paper rubrics to online tools for assessment, 
the teaching team of this course spent ten years researching, developing, 
improving, and reporting on assessment types (paper, verbal, online), assessment 
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groupings (self, peer, class, teacher), and tools (Moodle learning management 
system, forum module, VAM) blended in an EFL speech communication class, as 
outlined in supplementary materials, part A. Funding for the module development 
was provided by internal university grants and seed funding by universities in the 
Moodle Association of Japan. 


2.2. VAM functions 


Using the Moodle VAM, teachers or students could upload presentation videos 
directly to Moodle and students could watch them while completing self and/or 
peer assessments on predetermined rubrics, simple at first, and becoming more 
complex late in the course. Comments could also be made for each criteria and or 
as a whole. The teacher could also assess and separate weightings for self, peer, and 
teacher scores respectively. Figure | below shows a screen with a video-recording 
playback window for both teacher and student viewing while rubric scales are 
checked and brief comments are made. 


2.3. Data collection 


Both quantitative and qualitative data were collected in 2019, as done in previous 
cycles. Overall scores could be downloaded from the Moodle gradebook and the 
VAM directly and statistical analysis performed via Excel as shown in Table | 
below. 


In addition, end of course voluntary student satisfaction surveys (supplementary 
materials, part B) were given using the questionnaire module in Moodle. Qualitative 
data from students was also collected through the surveys and the Moodle Forum 
module helped collate qualitative data for the focus group of the teachers by 
recording weekly comments in a teaching journal. 


Table 1. Self, peer, teacher, and overall assessment average scores (out of 100) 
with teacher variances 


Presentation Average Scores (Presentations 1-5) 

2014 (n=55~63) 2017 (n=34~49) 2019 (n=34) 
Self 66.0 75.9 77.8 
Variance with teacher -8.8 -9.0 - 8.2 
Tie with teacher : i. Nib? : fe 
Teacher 74.9 84.8 86.3 


* In 2017, a timetable change required peer assessment to be dropped in order to reduce student workload 
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Figure 1. VAM rubric and comment feedback interface 
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3. Discussion 


Results from online learner assessment scores are consistent with those of previous 
years. Students continue to score themselves lower on post-performance assessment 
tasks than teachers on all presentations. An average total over the five presentations 
saw self assessments 8.2-9.0% lower than teacher assessments in the respective 
years, compared to only a 1.3-1.5 lower average difference by peers respectively. 
Students did not try to raise their score, but graded themselves more severely than 
their teachers. This is consistent with the general tendency of Japanese students to 
rate themselves modestly (Hinkelman & Cotter, 2018). Due to this high variance 
between teacher and self-ratings, a lower weighting of 20% was assigned to self 
assessment scores than to teacher scores (80%). 


From the 2019 student satisfaction surveys (supplementary materials, part B) we can 
see that 92% of students responded positively (agree or strongly agree) to watching 
their own videos, and 77% valued rating their own presentations respectively. 73% 
found value in classmates rating their presentations, which shows strong support 
for using the VAM tool for assessment and learning. 92% of students also regarded 
feedback from the teacher as helping them improve their presentations which could 
portray perceived teacher expertise, experience, or comparatively more detailed 
feedback by teachers compared to classmates on the rubric. Interestingly, the highest 
value of 96% was achieved on the survey by students agreeing that watching live 
presentations of their classmates was helpful to improving their presentations. 


4. Conclusions 


Over the ten cycles (years) of action research on this oral presentation course, the 
evolution of post-performance video watching, along with self and peer assessment, 
has proven to be a successful formative tool. This most recent 2019 cycle has been 
no different, results showing that the VAM draws the students into a more learner- 
focused mode of assessment, putting Gardner’s (2012) theory of ‘assessment for 
learning’ into practice. Students reported that being part of the assessment process 
through using the tool had helped them improve for future performances. Taking the 
role of ‘evaluators’ by using the VAM ultimately requires the students to first revisit 
the presentations again by viewing the videos, go through the cognitive process 
of scoring and giving feedback to their peers and to themselves, and finally reflect 
on all feedback received. We can also see that, as part of the assessment process, a 
complex rubric with specific criteria can be understood and used by intermediate- 
level students, in this case using their L2, to evaluate video-recorded student 
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performances in an oral presentation course. Although some cultural modesty took 
place, students placed enough importance on the task as not to try and purposefully 
score themselves or their peers higher than teachers, or wantonly assign grades due 
to lack of motivation or time. The convenience of the VAM being able to be used 
during class or out-of-class, and having the ability to create rubrics to match the 
assessment criteria and level of students, may have had a part to play in this. 


It is our view that future cycles of this research area need to concentrate on 
determining the most appropriate rubric language and rubric length to match 
learners and also to investigate whether students themselves have any ideas on 
how they would like to participate in the evaluation process. 
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